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THE EXPERTISE APPROACH AND SKILL 
ACQUISITION 

Research on expertise is by definition 
focused on a restricted sample of indi- 
viduals. Experts are people who consis- 
tently produce outstanding performance 
in their domains (Ericsson, 2006) and as 
such are without exception located on the 
positive side of the skill distribution. The 
usual approach in the study of exper- 
tise is to compare the extreme group of 
the skill distribution, experts, with the 
extreme group at the other end, that of 
novices. This contrasting approach, which 
we have called the "expertise approach" 
(Bilalic et al, 2010, 2012), has a long tra- 
dition (Chase and Simon, 1973; Simon 
and Chase, 1973; De Groot, 1978; Preacher 
et al, 2005). Its main advantage over the 
common approach in cognition, where all 
participants are at the same skill level, 
is the presence of a control group of 
novices that enables falsification of results 
obtained on experts (Wason, 1960; Kuhn, 
1970; Campitelli and Speelman, 2013). In 
that way, the expertise approach is not 
unlike the neuropsychological approach 
that contrasts results obtained on patients 
with the results of "normal" participants 
(Shallice, 1988). 

The main goal of the expertise 
approach is to provide evidence relating 
to the cognitive and neural mechanisms 
behind processes such as object and pat- 
tern recognition, which would be difficult 
to obtain from subjects who possess 
approximately the same level of expertise. 
The skill acquisition process, which is one 
of the main topics of expertise (William 
and Harter, 1899), is of secondary impor- 
tance in the expertise approach. This is 
understandable as the contrast between 



experts and novices captures only the 
beginning and the end product of the 
skill acquisition process. It is unrealistic 
to follow people for the length of time 
required in order to achieve expertise 
in a given domain. However, expertise 
researchers have recently started employ- 
ing an archival approach that provides a 
more complete picture of the skill acquisi- 
tion process (Charness and Gerchak, 1996; 
Chabris and Glickman, 2006; Howard, 
2008, 2009; Bilalic et al, 2009). In the 
game of chess, a domain commonly stud- 
ied in expertise research, there are precise 
records of all practitioners from an early 
age (Howard, 2006a; Bilalic et al, 2009). 
These records include not only personal 
information such as gender and age, but 
also skill levels at different stages, num- 
bers of games played, and corresponding 
skill change. The records provide a wealth 
of data for investigating the influence of 
factors such as age, gender, and even tal- 
ent, on the skill acquisition process. Here 
we want to draw attention to the fact 
that some of the databases used in pre- 
vious research only provide records of the 
very best practitioners. In the expertise 
approach, such restriction is an integral 
part of the methodology, but restricting 
the range of population in the archival 
approach could have grave consequences 
for the conclusions about the nature of 
skill acquisition. 

DIFFERENT DATABASES, DIFFERENT 
CONCLUSIONS 

One of the advantages of chess as a domain 
is that there is an objective and reliable 
measure of skill. Skill is measured on an 
interval scale that reflects the performance 
of a player against other players. The Elo 



rating, named after Arpad Elo who intro- 
duced the scale as a measure of chess skill 
(Elo, 1978), is measured in the same way 
all over the world. A beginner is supposed 
to have 600-800 Elo points, average play- 
ers about 1500 Elo points, master play- 
ers above 2200 Elo points, while the very 
best players, called grandmasters, have rat- 
ings above 2500 Elo points. Expert players 
are considered to have above 2000 rating 
points. 

The most frequently used database in 
skill acquisition studies is the database 
of the International Chess Federation, 
FIDE (for more information, see Howard, 
2006a). This database, like other chess 
databases we will mention here, offers 
multiple advantages for skill acquisition 
research. Firstly, it gathers records from the 
1970s to the present, and so it is possi- 
ble to obtain trajectories of ratings over 
the course of players' lives. Secondly, it 
represents the whole population of the 
very best players in the world. Thirdly, 
this database contains multiple measure- 
ment points from players, so it can be 
used to observe individual skill trajec- 
tories. Besides rating points, numerous 
other variables are recorded (e.g., num- 
ber of games played, gender of partic- 
ipants, nationality, title, rating change, 
rating rank) which could be used for 
research purposes (Howard, 2006a). In 
other words, the FIDE database offers a 
fruitful basis for exploration and descrip- 
tion of the multiple factors and processes 
behind chess skill acquisition. 

For all its advantages, the FIDE 
database provides only the records of the 
very best players. Due to technical and 
logistical reasons, the FIDE database at the 
beginning logged only master level players 
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(above 2200 Elo). Only in the 1990s was 
the level lowered to expert level players 
(2000 Elo) and then in the last decade to 
the level of average players (1500 Elo and 
below). In other words, the worst play- 
ers in the FIDE database are still average 
practitioners. 

The ideal situation would be to have 
records of all players from the very begin- 
ning of their careers, not only when they 
reach a particular level of expertise. In this 
scenario, the database would also encom- 
pass people who for whatever reasons do 
not become experts. Fortunately, there 
are such databases. National databases, 
such as the databases of the German 
Chess Federation and the United States 
Chess Federation (USCF), keep records 
of all their members and thus repre- 
sent the whole population of competi- 
tive (national) players. They provide all 
the information the FIDE database offers 
without restricting the range of skill. 

If one is trying to examine factors that 
influence skill acquisition, a database that 
contains only average and above-average 
players should not be the starting point 



of investigations. As is well known in 
the field of statistics, the conclusions 
obtained on data with restricted range 
could be misleading especially if they 
are not obtained by appropriate analy- 
ses (Long, 1997; Sackett and Yang, 2000). 
The best possible way to examine dif- 
ferences in effects that researchers could 
obtain while analyzing the data is through 
the comparison of effects made on differ- 
ent databases. Here we illustrate the pos- 
sible pitfalls of skill range restriction by 
comparing distributions of ratings from 
a database with restricted range (FIDE) 
and a database with unrestricted range 
(German). 

The FIDE and German databases con- 
tain similar number of practitioners, 
around 120,000 (see Figure 1). However, 
the ratings of the two distributions overlap 
only at the highest values of the German 
distribution and the lowest values of FIDE 
distribution. Not only are the mean and 
variance of both distribution vastly differ- 
ent, but also other parts of the distribu- 
tion, such as quartiles, are also extremely 
different. 



Restricting the database to the best 
players also has a consequence for the skill 
trajectories of players. One needs time to 
become an expert and the players in the 
FIDE database are older (37 years) than the 
players in the German database (32 years). 
The differences between the two databases 
are also evident when we compare typi- 
cal skill trajectories. Figure 2 A shows the 
FIDE players entering the database at 
around age 10, having already become 
competent players (rating of 1900 Elo), 
with a subsequent shallow increase to the 
peak at age 39. In contrast, the German 
players have a steeper increase, since they 
are entered the database as novices and 
learn faster at beginning skill level, as 
implied by the power law of practice 
(Newell and Rosenbloom, 1981), until the 
same peak at age 39. The decline in later 
years is also different in the two databases 
with FIDE players declining faster than 
their German counterparts. 

One could say that the FIDE and 
German players have vastly different rat- 
ings that make the comparison between 
them difficult. One way around this 
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FIGURE 1 | Distribution of chess skill as measured by Elo rating in 
FIDE (blue color) and German (red) databases (as of 2008). The 

databases contain similar number of players, but differ vastly in the 
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distribution shape and coverage — the only overlap is at the highest 
values of the German database and lowest values of the FIDE 
database. 
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FIGURE 2 | (A) Average skill trajectories, in Elo with 95% confidence intervals, over the years in FIDE database (blue color) and German (red) database. (B) 
Average standardized skill trajectories, in Z-values with 95% confidence intervals, over the years in FIDE database (blue color) and German (red) database. 



problem is to standardize ratings in each 
database separately and check if the skill 
trajectories are similar in both datasets. 
Figure 2B shows the standardized rating 
as a function of age. Again, on aver- 
age FIDE players start at a higher skill 
level but improve more slowly. They also 
have a higher peak but their decline is 
so rapid that in the latter stages of their 
careers their standardized performance is 
lower than the standardized performance 
of their German colleagues. 

EXPLAINING CONTRADICTORY 
RESULTS 

We have demonstrated that there are vast 
differences between the databases com- 
monly used in the archival approach to 
skill acquisition. The two datasets are com- 
pletely different in the range of values 
as well as in the number of participants 
that are obtaining a particular rating. The 
restricted databases, such as FIDE, do not 
represent the whole skill range and may 
provide inadequate answers to the ques- 
tions under investigation. The restriction 
of range and its consequences may also 
explain some of the inconsistencies and 
contradictory findings in the field. 

For example, Roring and Charness 
(2007) used the FIDE databases to inves- 
tigate age effects on skill acquisition. They 
demonstrated the peak age in chess skill 



to be around 43 years, much later than 
previously proposed peak around 35 years 
(e.g., Howard, 2005). Another surpris- 
ing result was the fact that the decline 
is steeper for initially lower rated par- 
ticipants than for higher rated partici- 
pants. In other words, initially more able 
participants were declining significantly 
more slowly than their initially weaker col- 
leagues. Our illustrations (Figures 2 A,B) 
indicate that both peak age and declining 
rate are influenced by the range restric- 
tion in the FIDE database. The conclu- 
sion would be significantly different if a 
whole range database, such as the German 
database, were used. 

To further illustrate possible conse- 
quences of the range restriction, we 
can consider the inconsistent findings 
in the research on gender differences 
in skill acquisition. It is notable that 
the studies using the restricted FIDE 
database regularly find gender differences 
in skill acquisition (Howard, 2005, 2006b, 
2014 but see, Bilalic and McLeod, 2006, 
2007). Furthermore, the studies using the 
national German and USCF databases 
(Chabris and Glickman, 2006; Bilalic et al., 
2009) also noted the differences in the 
mean and highest ratings of women and 
men. However, using the unrestricted 
range and the full lifespan data, they 
observed that factors such as participation 



rates and dropout rates could explain 
the differences. This kind of analysis is 
impossible with the FIDE database where 
dropouts are not recorded because the 
people concerned stopped playing chess 
before they achieved expert level. 

Researchers using the FIDE database 
to investigate talent (Howard, 2008, 2009) 
face a similar problem. The time required 
to reach a certain level and the amount 
of practice (as measured by games played) 
may well provide clues about the differ- 
ent natural endowments of certain players. 
This in turn may allow us to speculate 
about different levels of talent. It is, how- 
ever, impossible to make any certain con- 
clusions if we lack the very first part of 
their skill acquisition process, as we do 
in the FIDE database. As with the gen- 
der factor in skill acquisition process, the 
differences in the early stages may as well 
overshadow the differences at the highest 
level. Similarly, the causes behind dropouts 
may remain unresolved because the data 
of the people who for whatever reasons 
stopped playing chess is not available. 

Both the expertise and archival 
approaches are important vehicles for the 
investigation of expertise and cognition in 
general. The restricted range of focus in 
the expertise approach is a fundamental 
part of the methodology and an advan- 
tage over usual research on cognition. The 
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archival approach offers the possibility of 
capturing the full cycle of the long-term 
skill acquisition process. In this paper 
we have demonstrated that the results 
obtained on the restricted range do not 
necessarily generalize to the whole range of 
values. The effects obtained with restricted 
range cannot and should not be used to 
make inferences about the mechanisms 
and factors that influence skill acquisition. 
When we restrict our data, we restrict our 
conclusions too. 
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